In software engineering, testing is an integral part of the development cycle of an application or software. For this, testing an application or software involves generating test cases where each test case tests a certain functionality of the application or software. Often this involves invoking other aspects of the application or software. Code or scripts are written to develop each test case based on a specific test case logic/scenario. Further, scripts can be written to automate the testing process. However, writing individual testing logic for each test case often becomes a laborious and time consuming process. Furthermore, maintenance of automated test cases or test suites designed with traditional methods are also time consuming as any changes in functionality of Application Under Test (AUT) requires update/modification of multiple test case scripts.
FIG. 1 is a block diagram illustrating automated testing of an AUT using known traditional methods. Framework 102 includes framework engine 104 interacting with the framework library 106. Framework library 106 includes functions or procedures provided within a framework function library 108 where all the functions needed to test an AUT are provided. Test repository 110 includes one or more test suites 112. Each test suite 112 includes scripts/code logic for one or more test cases 114, each test case 114 testing a functionality or aspect of the AUT. In traditional methods, function library 106 is directly used (called) from test case 114 script of a test suite 112.
Framework 102 represents a general test automation framework including three main components, framework engine 104, framework library 106 (representing a collection of framework function library 108) and a test repository 110. The framework engine 104 represents the main script that configures environment for execution, runs the test suites 112 which represents a sequence of test cases 114, and is responsible for the output results or reports of test execution. The framework library 106 includes framework function library 108 which represents a collection of scripts that cover general functions or procedures required by the test cases/scenarios (e.g. data transformation, search processing, common command execution with parameters, etc.). With traditional test automation methods these functional scripts are utilized within test case's scripts to partially eliminate redundancy of the code.
The test repository 110 represents a collection of automated test cases usually grouped by test suites 112 to assist in maintenance. A test suite 112 represents a container for test cases 114 that are used for testing specific functionality, or a specific aspect of the AUT. Usually, grouping test cases 114 in test suite 112 helps in maintenance and planning for test coverage. However, such maintenance and planning can be limited in large scale application development if a functionality or aspect of the AUT changes during development.
FIG. 2 is a block diagram illustrating the problem associated with traditional methods. Traditional test suite 200 shows test suite 200 including two test cases, test case 1 202 and test case 2 204. In the test suite 200, test case 1 202 and test case 2 204 are grouped together in one suite because both test at least the same aspects of the AUT (in this case feature A of the AUT as disclosed below). Test case 1 202 and test case 2 204 of test suite 200 tests hypothetical features A, B, C, X, and Y of an AUT. It should be noted variables are used instead of actual testing features as the underlying problem associated with the traditional testing methods can be readily noticed without providing exact detail of the described features. In this hypothetical we presume features A, B, C, X, or Y each represents any feature or aspect of the AUT. It is presumed a successful testing of a given feature would return a logical value of ‘true’ or a value representing a logical value of ‘true’, and a failure of a given feature would return a logical value of ‘false’ or a value representing a logical value of ‘false’. Test suite 200 represents a test suite in test repository 110. All test case(s) 114 in test repository 110 are written using framework library 106 and use framework engine 104 to process the AUT.
As an example, in FIG. 2, multiple test cases 114 of one test suite 112 are represented as test case 1 202 and test case 2 204 as represented by test suite 200. In this example, test case 1 202 tests the functionality of hypothetical features A, B, and C. Under traditional methods, test case 1 202 includes logic/code to shutdown and restart a service represented by the AUT to ensure memory has been cleared. Test case 1 202 further includes logic/code to test the functionality of hypothetical feature A of the AUT. Further, test case 1 202 includes logic/code to test the functionality of hypothetical feature B of the AUT, where feature B of the AUT is dependent on success of feature A. As explained above, a success of a feature means that a logical value of ‘true’ or a value representing a logical value of ‘true’ is a requirement to test dependent feature B of the AUT. Furthermore, test case 1 202 includes logic/code to test the functionality of feature C of the AUT where feature C is also dependent on the success of feature A.
As illustrated in FIG. 2, test suite 200 also includes test case 2 204 which tests hypothetical features A, X, and Y of the AUT. Similar to test case 1 202 test case 2 204 includes logic/code to shutdown and restart the service represented by the AUT to ensure memory has been cleared. Furthermore, similar to test case 1 202, test case 2 204 includes logic/code to test the functionality of feature A of the AUT. Further test case 2 204 has logic/code to test the functionality of feature X, where feature X is dependent on the success of feature A. Similarly, test case 204 has code to test the functionality of feature Y of the AUT, where feature Y is dependent on the success of feature X.
As can be seen both test cases, test case 1 202 and test case 2 204, have common code/logic to shutdown and restart a service represented by the AUT as well as testing the functionality of feature A. Under traditional testing methods, all code/logic described above would need to be written separately for both test case 1 202, and test case 2 204, including the common code/logic required for each test case. Thus, any change in the shutdown/restart code or in functionality A, during development, would require changing the code/logic in both test cases separately. Such a scenario can be problematic in large scale application development where features are often modified during the development phase of a project. Thus, using traditional testing methods, maintenance of automated test cases can be problematic and time consuming as maintenance of traditional automated test suites would require changes in functionality of the AUT that results in update/modification of multiple test cases scripts.
This problem can be exponentially increased in multi-tiered, large scale application development. Thus, what is needed is an efficient and scalable method of accelerated test automation of the application or software.